{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Kubeflow Pipelines with Katib component\n",
    "\n",
    "In this notebook you will:\n",
    "- Create Katib Experiment using random algorithm.\n",
    "- Use median stopping rule as an early stopping algorithm.\n",
    "- Use Kubernetes Job with mxnet mnist training container as a Trial template.\n",
    "- Create Pipeline to get the optimal hyperparameters.\n",
    "\n",
    "Reference documentation:\n",
    "- https://kubeflow.org/docs/components/katib/experiment/#random-search\n",
    "- https://kubeflow.org/docs/components/katib/early-stopping/\n",
    "- https://kubeflow.org/docs/pipelines/overview/concepts/component/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Install required package\n",
    "\n",
    "Kubeflow Pipelines SDK and Kubeflow Katib SDK."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Defaulting to user installation because normal site-packages is not writeable\n",
      "Collecting pip\n",
      "  Downloading pip-20.3-py2.py3-none-any.whl (1.5 MB)\n",
      "\u001b[K     |████████████████████████████████| 1.5 MB 30.4 MB/s eta 0:00:01\n",
      "\u001b[?25hInstalling collected packages: pip\n",
      "\u001b[33m  WARNING: The scripts pip, pip3 and pip3.6 are installed in '/home/jovyan/.local/bin' which is not on PATH.\n",
      "  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\u001b[0m\n",
      "Successfully installed pip-20.3\n",
      "\u001b[33mWARNING: You are using pip version 20.0.2; however, version 20.3 is available.\n",
      "You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.\u001b[0m\n",
      "Defaulting to user installation because normal site-packages is not writeable\n",
      "Collecting kfp==1.1.1\n",
      "  Downloading kfp-1.1.1.tar.gz (162 kB)\n",
      "\u001b[K     |████████████████████████████████| 162 kB 16.2 MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: PyYAML in /usr/local/lib/python3.6/dist-packages (from kfp==1.1.1) (5.3)\n",
      "Requirement already satisfied: google-cloud-storage>=1.13.0 in /usr/local/lib/python3.6/dist-packages (from kfp==1.1.1) (1.25.0)\n",
      "Requirement already satisfied: kubernetes<12.0.0,>=8.0.0 in /usr/local/lib/python3.6/dist-packages (from kfp==1.1.1) (10.0.1)\n",
      "Requirement already satisfied: google-auth>=1.6.1 in /usr/local/lib/python3.6/dist-packages (from kfp==1.1.1) (1.11.0)\n",
      "Requirement already satisfied: cloudpickle in /usr/local/lib/python3.6/dist-packages (from kfp==1.1.1) (1.2.2)\n",
      "Requirement already satisfied: jsonschema>=3.0.1 in /usr/local/lib/python3.6/dist-packages (from kfp==1.1.1) (3.2.0)\n",
      "Collecting click\n",
      "  Downloading click-7.1.2-py2.py3-none-any.whl (82 kB)\n",
      "\u001b[K     |████████████████████████████████| 82 kB 1.2 MB/s  eta 0:00:01\n",
      "\u001b[?25hCollecting Deprecated\n",
      "  Downloading Deprecated-1.2.10-py2.py3-none-any.whl (8.7 kB)\n",
      "Requirement already satisfied: wrapt<2,>=1.10 in /usr/local/lib/python3.6/dist-packages (from Deprecated->kfp==1.1.1) (1.11.2)\n",
      "Collecting docstring-parser>=0.7.3\n",
      "  Downloading docstring_parser-0.7.3.tar.gz (13 kB)\n",
      "  Installing build dependencies ... \u001b[?25ldone\n",
      "\u001b[?25h  Getting requirements to build wheel ... \u001b[?25ldone\n",
      "\u001b[?25h    Preparing wheel metadata ... \u001b[?25ldone\n",
      "\u001b[?25hRequirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.6/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (0.2.8)\n",
      "Requirement already satisfied: six>=1.9.0 in /usr/lib/python3/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (1.11.0)\n",
      "Requirement already satisfied: setuptools>=40.3.0 in /usr/local/lib/python3.6/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (45.1.0)\n",
      "Requirement already satisfied: rsa<4.1,>=3.1.4 in /usr/local/lib/python3.6/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (4.0)\n",
      "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (4.0.0)\n",
      "Requirement already satisfied: google-auth>=1.6.1 in /usr/local/lib/python3.6/dist-packages (from kfp==1.1.1) (1.11.0)\n",
      "Requirement already satisfied: google-resumable-media<0.6dev,>=0.5.0 in /usr/local/lib/python3.6/dist-packages (from google-cloud-storage>=1.13.0->kfp==1.1.1) (0.5.0)\n",
      "Requirement already satisfied: google-cloud-core<2.0dev,>=1.2.0 in /usr/local/lib/python3.6/dist-packages (from google-cloud-storage>=1.13.0->kfp==1.1.1) (1.3.0)\n",
      "Requirement already satisfied: google-api-core<2.0.0dev,>=1.16.0 in /usr/local/lib/python3.6/dist-packages (from google-cloud-core<2.0dev,>=1.2.0->google-cloud-storage>=1.13.0->kfp==1.1.1) (1.16.0)\n",
      "Requirement already satisfied: pytz in /usr/local/lib/python3.6/dist-packages (from google-api-core<2.0.0dev,>=1.16.0->google-cloud-core<2.0dev,>=1.2.0->google-cloud-storage>=1.13.0->kfp==1.1.1) (2019.3)\n",
      "Requirement already satisfied: six>=1.9.0 in /usr/lib/python3/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (1.11.0)\n",
      "Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /usr/local/lib/python3.6/dist-packages (from google-api-core<2.0.0dev,>=1.16.0->google-cloud-core<2.0dev,>=1.2.0->google-cloud-storage>=1.13.0->kfp==1.1.1) (1.51.0)\n",
      "Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /usr/local/lib/python3.6/dist-packages (from google-api-core<2.0.0dev,>=1.16.0->google-cloud-core<2.0dev,>=1.2.0->google-cloud-storage>=1.13.0->kfp==1.1.1) (2.22.0)\n",
      "Requirement already satisfied: setuptools>=40.3.0 in /usr/local/lib/python3.6/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (45.1.0)\n",
      "Requirement already satisfied: protobuf>=3.4.0 in /usr/local/lib/python3.6/dist-packages (from google-api-core<2.0.0dev,>=1.16.0->google-cloud-core<2.0dev,>=1.2.0->google-cloud-storage>=1.13.0->kfp==1.1.1) (3.11.2)\n",
      "Requirement already satisfied: google-auth>=1.6.1 in /usr/local/lib/python3.6/dist-packages (from kfp==1.1.1) (1.11.0)\n",
      "Requirement already satisfied: six>=1.9.0 in /usr/lib/python3/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (1.11.0)\n",
      "Requirement already satisfied: protobuf>=3.4.0 in /usr/local/lib/python3.6/dist-packages (from google-api-core<2.0.0dev,>=1.16.0->google-cloud-core<2.0dev,>=1.2.0->google-cloud-storage>=1.13.0->kfp==1.1.1) (3.11.2)\n",
      "Requirement already satisfied: six>=1.9.0 in /usr/lib/python3/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (1.11.0)\n",
      "Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.6/dist-packages (from jsonschema>=3.0.1->kfp==1.1.1) (19.3.0)\n",
      "Requirement already satisfied: setuptools>=40.3.0 in /usr/local/lib/python3.6/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (45.1.0)\n",
      "Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.6/dist-packages (from jsonschema>=3.0.1->kfp==1.1.1) (1.4.0)\n",
      "Requirement already satisfied: pyrsistent>=0.14.0 in /usr/local/lib/python3.6/dist-packages (from jsonschema>=3.0.1->kfp==1.1.1) (0.15.7)\n",
      "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata->jsonschema>=3.0.1->kfp==1.1.1) (2.1.0)\n",
      "Collecting kfp-pipeline-spec<0.2.0,>=0.1.0\n",
      "  Downloading kfp_pipeline_spec-0.1.2-py3-none-any.whl (21 kB)\n",
      "Collecting kfp-server-api<2.0.0,>=0.2.5\n",
      "  Downloading kfp-server-api-1.0.4.tar.gz (51 kB)\n",
      "\u001b[K     |████████████████████████████████| 51 kB 945 kB/s  eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: urllib3>=1.15 in /usr/local/lib/python3.6/dist-packages (from kfp-server-api<2.0.0,>=0.2.5->kfp==1.1.1) (1.25.8)\n",
      "Requirement already satisfied: six>=1.9.0 in /usr/lib/python3/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (1.11.0)\n",
      "Requirement already satisfied: certifi in /usr/local/lib/python3.6/dist-packages (from kfp-server-api<2.0.0,>=0.2.5->kfp==1.1.1) (2019.11.28)\n",
      "Requirement already satisfied: python-dateutil in /usr/local/lib/python3.6/dist-packages (from kfp-server-api<2.0.0,>=0.2.5->kfp==1.1.1) (2.8.1)\n",
      "Requirement already satisfied: requests-oauthlib in /usr/local/lib/python3.6/dist-packages (from kubernetes<12.0.0,>=8.0.0->kfp==1.1.1) (1.3.0)\n",
      "Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /usr/local/lib/python3.6/dist-packages (from google-api-core<2.0.0dev,>=1.16.0->google-cloud-core<2.0dev,>=1.2.0->google-cloud-storage>=1.13.0->kfp==1.1.1) (2.22.0)\n",
      "Requirement already satisfied: certifi in /usr/local/lib/python3.6/dist-packages (from kfp-server-api<2.0.0,>=0.2.5->kfp==1.1.1) (2019.11.28)\n",
      "Requirement already satisfied: PyYAML in /usr/local/lib/python3.6/dist-packages (from kfp==1.1.1) (5.3)\n",
      "Requirement already satisfied: urllib3>=1.15 in /usr/local/lib/python3.6/dist-packages (from kfp-server-api<2.0.0,>=0.2.5->kfp==1.1.1) (1.25.8)\n",
      "Requirement already satisfied: setuptools>=40.3.0 in /usr/local/lib/python3.6/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (45.1.0)\n",
      "Requirement already satisfied: google-auth>=1.6.1 in /usr/local/lib/python3.6/dist-packages (from kfp==1.1.1) (1.11.0)\n",
      "Requirement already satisfied: websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 in /usr/local/lib/python3.6/dist-packages (from kubernetes<12.0.0,>=8.0.0->kfp==1.1.1) (0.57.0)\n",
      "Requirement already satisfied: six>=1.9.0 in /usr/lib/python3/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (1.11.0)\n",
      "Requirement already satisfied: python-dateutil in /usr/local/lib/python3.6/dist-packages (from kfp-server-api<2.0.0,>=0.2.5->kfp==1.1.1) (2.8.1)\n",
      "Requirement already satisfied: six>=1.9.0 in /usr/lib/python3/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (1.11.0)\n",
      "Requirement already satisfied: setuptools>=40.3.0 in /usr/local/lib/python3.6/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (45.1.0)\n",
      "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.6/dist-packages (from pyasn1-modules>=0.2.1->google-auth>=1.6.1->kfp==1.1.1) (0.4.8)\n",
      "Requirement already satisfied: six>=1.9.0 in /usr/lib/python3/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (1.11.0)\n",
      "Requirement already satisfied: six>=1.9.0 in /usr/lib/python3/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (1.11.0)\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.16.0->google-cloud-core<2.0dev,>=1.2.0->google-cloud-storage>=1.13.0->kfp==1.1.1) (3.0.4)\n",
      "Requirement already satisfied: certifi in /usr/local/lib/python3.6/dist-packages (from kfp-server-api<2.0.0,>=0.2.5->kfp==1.1.1) (2019.11.28)\n",
      "Requirement already satisfied: urllib3>=1.15 in /usr/local/lib/python3.6/dist-packages (from kfp-server-api<2.0.0,>=0.2.5->kfp==1.1.1) (1.25.8)\n",
      "Requirement already satisfied: idna<2.9,>=2.5 in /usr/lib/python3/dist-packages (from requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.16.0->google-cloud-core<2.0dev,>=1.2.0->google-cloud-storage>=1.13.0->kfp==1.1.1) (2.6)\n",
      "Requirement already satisfied: requests<3.0.0dev,>=2.18.0 in /usr/local/lib/python3.6/dist-packages (from google-api-core<2.0.0dev,>=1.16.0->google-cloud-core<2.0dev,>=1.2.0->google-cloud-storage>=1.13.0->kfp==1.1.1) (2.22.0)\n",
      "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.6/dist-packages (from requests-oauthlib->kubernetes<12.0.0,>=8.0.0->kfp==1.1.1) (3.1.0)\n",
      "Collecting requests_toolbelt>=0.8.0\n",
      "  Downloading requests_toolbelt-0.9.1-py2.py3-none-any.whl (54 kB)\n",
      "\u001b[K     |████████████████████████████████| 54 kB 3.7 MB/s  eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: requests<3.0.0dev,>=2.18.0 in /usr/local/lib/python3.6/dist-packages (from google-api-core<2.0.0dev,>=1.16.0->google-cloud-core<2.0dev,>=1.2.0->google-cloud-storage>=1.13.0->kfp==1.1.1) (2.22.0)\n",
      "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.6/dist-packages (from pyasn1-modules>=0.2.1->google-auth>=1.6.1->kfp==1.1.1) (0.4.8)\n",
      "Collecting strip-hints\n",
      "  Downloading strip-hints-0.1.9.tar.gz (30 kB)\n",
      "Requirement already satisfied: wheel in /usr/lib/python3/dist-packages (from strip-hints->kfp==1.1.1) (0.30.0)\n",
      "Collecting tabulate\n",
      "  Downloading tabulate-0.8.7-py3-none-any.whl (24 kB)\n",
      "Requirement already satisfied: six>=1.9.0 in /usr/lib/python3/dist-packages (from google-auth>=1.6.1->kfp==1.1.1) (1.11.0)\n",
      "Building wheels for collected packages: kfp, docstring-parser, kfp-server-api, strip-hints\n",
      "  Building wheel for kfp (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for kfp: filename=kfp-1.1.1-py3-none-any.whl size=222518 sha256=a3a3530489f3ca42e25ad6a221909142725d94df20b45106775801e271807ecb\n",
      "  Stored in directory: /home/jovyan/.cache/pip/wheels/88/9f/b1/149db749eeb26a01ddc590b68d2b9c27a9c7eedb799f47d9f1\n",
      "  Building wheel for docstring-parser (PEP 517) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for docstring-parser: filename=docstring_parser-0.7.3-py3-none-any.whl size=19230 sha256=df43b4eb88d8f80a3c47f9aa2ecd07ddbfcbead920ed2697911659cd9f185a64\n",
      "  Stored in directory: /home/jovyan/.cache/pip/wheels/32/63/02/bb6eebc5261f10a6de3dcf26336a7b2b8b8dc8cacb6c00f75f\n",
      "  Building wheel for kfp-server-api (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for kfp-server-api: filename=kfp_server_api-1.0.4-py3-none-any.whl size=105034 sha256=74c422f07d14ea87395dc7d3287076bf79f9391bcbc076171ea7afeb43ed77bc\n",
      "  Stored in directory: /home/jovyan/.cache/pip/wheels/0f/50/2d/28c9ae498b2e5ff5bf5a9765bca3dcd08ab4a79333670de6cb\n",
      "  Building wheel for strip-hints (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for strip-hints: filename=strip_hints-0.1.9-py2.py3-none-any.whl size=24671 sha256=33ada8d9d7a77196257d7b1205856c86ae7390df64b65dba4911c42895b568e4\n",
      "  Stored in directory: /home/jovyan/.cache/pip/wheels/21/6d/fa/7ed7c0560e1ef39ebabd5cc0241e7fca711660bae1ad752e2b\n",
      "Successfully built kfp docstring-parser kfp-server-api strip-hints\n",
      "Installing collected packages: tabulate, strip-hints, requests-toolbelt, kfp-server-api, kfp-pipeline-spec, docstring-parser, Deprecated, click, kfp\n",
      "\u001b[33m  WARNING: The script tabulate is installed in '/home/jovyan/.local/bin' which is not on PATH.\n",
      "  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\u001b[0m\n",
      "\u001b[33m  WARNING: The script strip-hints is installed in '/home/jovyan/.local/bin' which is not on PATH.\n",
      "  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\u001b[0m\n",
      "\u001b[33m  WARNING: The scripts dsl-compile, dsl-compile-v2 and kfp are installed in '/home/jovyan/.local/bin' which is not on PATH.\n",
      "  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.\u001b[0m\n",
      "Successfully installed Deprecated-1.2.10 click-7.1.2 docstring-parser-0.7.3 kfp-1.1.1 kfp-pipeline-spec-0.1.2 kfp-server-api-1.0.4 requests-toolbelt-0.9.1 strip-hints-0.1.9 tabulate-0.8.7\n",
      "Defaulting to user installation because normal site-packages is not writeable\n",
      "Collecting kubeflow-katib==0.10.1\n",
      "  Downloading kubeflow_katib-0.10.1-py3-none-any.whl (113 kB)\n",
      "\u001b[K     |████████████████████████████████| 113 kB 30.1 MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: kubernetes==10.0.1 in /usr/local/lib/python3.6/dist-packages (from kubeflow-katib==0.10.1) (10.0.1)\n",
      "Requirement already satisfied: urllib3>=1.15.1 in /usr/local/lib/python3.6/dist-packages (from kubeflow-katib==0.10.1) (1.25.8)\n",
      "Requirement already satisfied: setuptools>=21.0.0 in /usr/local/lib/python3.6/dist-packages (from kubeflow-katib==0.10.1) (45.1.0)\n",
      "Requirement already satisfied: certifi>=14.05.14 in /usr/local/lib/python3.6/dist-packages (from kubeflow-katib==0.10.1) (2019.11.28)\n",
      "Requirement already satisfied: python-dateutil>=2.5.3 in /usr/local/lib/python3.6/dist-packages (from kubeflow-katib==0.10.1) (2.8.1)\n",
      "Requirement already satisfied: six>=1.10 in /usr/lib/python3/dist-packages (from kubeflow-katib==0.10.1) (1.11.0)\n",
      "Requirement already satisfied: six>=1.10 in /usr/lib/python3/dist-packages (from kubeflow-katib==0.10.1) (1.11.0)\n",
      "Requirement already satisfied: requests-oauthlib in /usr/local/lib/python3.6/dist-packages (from kubernetes==10.0.1->kubeflow-katib==0.10.1) (1.3.0)\n",
      "Requirement already satisfied: urllib3>=1.15.1 in /usr/local/lib/python3.6/dist-packages (from kubeflow-katib==0.10.1) (1.25.8)\n",
      "Requirement already satisfied: setuptools>=21.0.0 in /usr/local/lib/python3.6/dist-packages (from kubeflow-katib==0.10.1) (45.1.0)\n",
      "Requirement already satisfied: pyyaml>=3.12 in /usr/local/lib/python3.6/dist-packages (from kubernetes==10.0.1->kubeflow-katib==0.10.1) (5.3)\n",
      "Requirement already satisfied: certifi>=14.05.14 in /usr/local/lib/python3.6/dist-packages (from kubeflow-katib==0.10.1) (2019.11.28)\n",
      "Requirement already satisfied: python-dateutil>=2.5.3 in /usr/local/lib/python3.6/dist-packages (from kubeflow-katib==0.10.1) (2.8.1)\n",
      "Requirement already satisfied: websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 in /usr/local/lib/python3.6/dist-packages (from kubernetes==10.0.1->kubeflow-katib==0.10.1) (0.57.0)\n",
      "Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from kubernetes==10.0.1->kubeflow-katib==0.10.1) (2.22.0)\n",
      "Requirement already satisfied: google-auth>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from kubernetes==10.0.1->kubeflow-katib==0.10.1) (1.11.0)\n",
      "Requirement already satisfied: six>=1.10 in /usr/lib/python3/dist-packages (from kubeflow-katib==0.10.1) (1.11.0)\n",
      "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.6/dist-packages (from google-auth>=1.0.1->kubernetes==10.0.1->kubeflow-katib==0.10.1) (0.2.8)\n",
      "Requirement already satisfied: rsa<4.1,>=3.1.4 in /usr/local/lib/python3.6/dist-packages (from google-auth>=1.0.1->kubernetes==10.0.1->kubeflow-katib==0.10.1) (4.0)\n",
      "Requirement already satisfied: setuptools>=21.0.0 in /usr/local/lib/python3.6/dist-packages (from kubeflow-katib==0.10.1) (45.1.0)\n",
      "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from google-auth>=1.0.1->kubernetes==10.0.1->kubeflow-katib==0.10.1) (4.0.0)\n",
      "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.6/dist-packages (from pyasn1-modules>=0.2.1->google-auth>=1.0.1->kubernetes==10.0.1->kubeflow-katib==0.10.1) (0.4.8)\n",
      "Requirement already satisfied: six>=1.10 in /usr/lib/python3/dist-packages (from kubeflow-katib==0.10.1) (1.11.0)\n",
      "Requirement already satisfied: certifi>=14.05.14 in /usr/local/lib/python3.6/dist-packages (from kubeflow-katib==0.10.1) (2019.11.28)\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->kubernetes==10.0.1->kubeflow-katib==0.10.1) (3.0.4)\n",
      "Requirement already satisfied: idna<2.9,>=2.5 in /usr/lib/python3/dist-packages (from requests->kubernetes==10.0.1->kubeflow-katib==0.10.1) (2.6)\n",
      "Requirement already satisfied: urllib3>=1.15.1 in /usr/local/lib/python3.6/dist-packages (from kubeflow-katib==0.10.1) (1.25.8)\n",
      "Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from kubernetes==10.0.1->kubeflow-katib==0.10.1) (2.22.0)\n",
      "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.6/dist-packages (from requests-oauthlib->kubernetes==10.0.1->kubeflow-katib==0.10.1) (3.1.0)\n",
      "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.6/dist-packages (from pyasn1-modules>=0.2.1->google-auth>=1.0.1->kubernetes==10.0.1->kubeflow-katib==0.10.1) (0.4.8)\n",
      "Collecting table-logger>=0.3.5\n",
      "  Downloading table_logger-0.3.6-py3-none-any.whl (14 kB)\n",
      "Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from table-logger>=0.3.5->kubeflow-katib==0.10.1) (1.18.1)\n",
      "Requirement already satisfied: six>=1.10 in /usr/lib/python3/dist-packages (from kubeflow-katib==0.10.1) (1.11.0)\n",
      "Installing collected packages: table-logger, kubeflow-katib\n",
      "Successfully installed kubeflow-katib-0.10.1 table-logger-0.3.6\n"
     ]
    }
   ],
   "source": [
    "# Update the PIP version.\n",
    "!python -m pip install --upgrade pip\n",
    "!pip install kfp==1.1.1\n",
    "!pip install kubeflow-katib==0.10.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Restart the Notebook kernel to use the SDK packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<script>Jupyter.notebook.kernel.restart()</script>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.display import display_html\n",
    "display_html(\"<script>Jupyter.notebook.kernel.restart()</script>\",raw=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import required packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import kfp\n",
    "import kfp.dsl as dsl\n",
    "from kfp import components\n",
    "\n",
    "from kubeflow.katib import ApiClient\n",
    "from kubeflow.katib import V1beta1ExperimentSpec\n",
    "from kubeflow.katib import V1beta1AlgorithmSpec\n",
    "from kubeflow.katib import V1beta1EarlyStoppingSpec\n",
    "from kubeflow.katib import V1beta1EarlyStoppingSetting\n",
    "from kubeflow.katib import V1beta1ObjectiveSpec\n",
    "from kubeflow.katib import V1beta1ParameterSpec\n",
    "from kubeflow.katib import V1beta1FeasibleSpace\n",
    "from kubeflow.katib import V1beta1TrialTemplate\n",
    "from kubeflow.katib import V1beta1TrialParameterSpec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define an Experiment\n",
    "\n",
    "You have to create an Experiment object before deploying it. This Experiment is similar to [this](https://github.com/kubeflow/katib/blob/master/examples/v1beta1/early-stopping/median-stop.yaml) YAML."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Experiment name and namespace.\n",
    "experiment_name = \"median-stop\"\n",
    "experiment_namespace = \"anonymous\"\n",
    "\n",
    "# Trial count specification.\n",
    "max_trial_count = 18\n",
    "max_failed_trial_count = 3\n",
    "parallel_trial_count = 2\n",
    "\n",
    "# Objective specification.\n",
    "objective=V1beta1ObjectiveSpec(\n",
    "    type=\"maximize\",\n",
    "    goal= 0.99,\n",
    "    objective_metric_name=\"Validation-accuracy\",\n",
    "    additional_metric_names=[\n",
    "        \"Train-accuracy\"\n",
    "    ]\n",
    ")\n",
    "\n",
    "# Algorithm specification.\n",
    "algorithm=V1beta1AlgorithmSpec(\n",
    "    algorithm_name=\"random\",\n",
    ")\n",
    "\n",
    "# Early Stopping specification.\n",
    "early_stopping=V1beta1EarlyStoppingSpec(\n",
    "    algorithm_name=\"medianstop\",\n",
    "    algorithm_settings=[\n",
    "        V1beta1EarlyStoppingSetting(\n",
    "            name=\"min_trials_required\",\n",
    "            value=\"2\"\n",
    "        )\n",
    "    ]\n",
    ")\n",
    "\n",
    "\n",
    "# Experiment search space.\n",
    "# In this example we tune learning rate, number of layer and optimizer.\n",
    "# Learning rate has bad feasible space to show more early stopped Trials.\n",
    "parameters=[\n",
    "    V1beta1ParameterSpec(\n",
    "        name=\"lr\",\n",
    "        parameter_type=\"double\",\n",
    "        feasible_space=V1beta1FeasibleSpace(\n",
    "            min=\"0.01\",\n",
    "            max=\"0.3\"\n",
    "        ),\n",
    "    ),\n",
    "    V1beta1ParameterSpec(\n",
    "        name=\"num-layers\",\n",
    "        parameter_type=\"int\",\n",
    "        feasible_space=V1beta1FeasibleSpace(\n",
    "            min=\"2\",\n",
    "            max=\"5\"\n",
    "        ),\n",
    "    ),\n",
    "    V1beta1ParameterSpec(\n",
    "        name=\"optimizer\",\n",
    "        parameter_type=\"categorical\",\n",
    "        feasible_space=V1beta1FeasibleSpace(\n",
    "            list=[\n",
    "                \"sgd\", \n",
    "                \"adam\",\n",
    "                \"ftrl\"\n",
    "            ]\n",
    "        ),\n",
    "    ),\n",
    "]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define a Trial template\n",
    "\n",
    "In this example, the Trial's Worker is the Kubernetes Job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# JSON template specification for the Trial's Worker Kubernetes Job.\n",
    "trial_spec={\n",
    "    \"apiVersion\": \"batch/v1\",\n",
    "    \"kind\": \"Job\",\n",
    "    \"spec\": {\n",
    "        \"template\": {\n",
    "            \"metadata\": {\n",
    "                \"annotations\": {\n",
    "                     \"sidecar.istio.io/inject\": \"false\"\n",
    "                }\n",
    "            },\n",
    "            \"spec\": {\n",
    "                \"containers\": [\n",
    "                    {\n",
    "                        \"name\": \"training-container\",\n",
    "                        \"image\": \"docker.io/kubeflowkatib/mxnet-mnist:v1beta1-e294a90\",\n",
    "                        \"command\": [\n",
    "                            \"python3\",\n",
    "                            \"/opt/mxnet-mnist/mnist.py\",\n",
    "                            \"--batch-size=64\",\n",
    "                            \"--lr=${trialParameters.learningRate}\",\n",
    "                            \"--num-layers=${trialParameters.numberLayers}\",\n",
    "                            \"--optimizer=${trialParameters.optimizer}\"\n",
    "                        ]\n",
    "                    }\n",
    "                ],\n",
    "                \"restartPolicy\": \"Never\"\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "}\n",
    "\n",
    "# Configure parameters for the Trial template.\n",
    "# We set the retain parameter to \"True\" to not clean-up the Trial Job's Kubernetes Pods.\n",
    "trial_template=V1beta1TrialTemplate(\n",
    "    retain=True,\n",
    "    primary_container_name=\"training-container\",\n",
    "    trial_parameters=[\n",
    "        V1beta1TrialParameterSpec(\n",
    "            name=\"learningRate\",\n",
    "            description=\"Learning rate for the training model\",\n",
    "            reference=\"lr\"\n",
    "        ),\n",
    "        V1beta1TrialParameterSpec(\n",
    "            name=\"numberLayers\",\n",
    "            description=\"Number of training model layers\",\n",
    "            reference=\"num-layers\"\n",
    "        ),\n",
    "        V1beta1TrialParameterSpec(\n",
    "            name=\"optimizer\",\n",
    "            description=\"Training model optimizer (sdg, adam or ftrl)\",\n",
    "            reference=\"optimizer\"\n",
    "        ),\n",
    "    ],\n",
    "    trial_spec=trial_spec\n",
    ")\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define an Experiment specification\n",
    "\n",
    "Create an Experiment specification from the above parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "experiment_spec=V1beta1ExperimentSpec(\n",
    "    max_trial_count=max_trial_count,\n",
    "    max_failed_trial_count=max_failed_trial_count,\n",
    "    parallel_trial_count=parallel_trial_count,\n",
    "    objective=objective,\n",
    "    algorithm=algorithm,\n",
    "    early_stopping=early_stopping,\n",
    "    parameters=parameters,\n",
    "    trial_template=trial_template\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create a Pipeline using Katib component\n",
    "\n",
    "The best hyperparameters are printed after Experiment is finished.\n",
    "The Experiment is not deleted after the Pipeline is finished."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the Katib launcher.\n",
    "katib_experiment_launcher_op = components.load_component_from_url(\n",
    "    \"https://raw.githubusercontent.com/kubeflow/pipelines/master/components/kubeflow/katib-launcher/component.yaml\")\n",
    "\n",
    "@dsl.pipeline(\n",
    "    name=\"Launch Katib early stopping Experiment\",\n",
    "    description=\"An example to launch Katib Experiment with early stopping\"\n",
    ")\n",
    "\n",
    "def median_stop():\n",
    "    \n",
    "    # Katib launcher component.\n",
    "    # Experiment Spec should be serialized to a valid Kubernetes object.\n",
    "    op = katib_experiment_launcher_op(\n",
    "        experiment_name=experiment_name,\n",
    "        experiment_namespace=experiment_namespace,\n",
    "        experiment_spec=ApiClient().sanitize_for_serialization(experiment_spec),\n",
    "        experiment_timeout_minutes=60,\n",
    "        delete_finished_experiment=False)\n",
    "    \n",
    "    # Output container to print the results.\n",
    "    op_out = dsl.ContainerOp(\n",
    "        name=\"best-hp\",\n",
    "        image=\"library/bash:4.4.23\",\n",
    "        command=[\"sh\", \"-c\"],\n",
    "        arguments=[\"echo Best HyperParameters: %s\" % op.output],\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run the Pipeline\n",
    "\n",
    "You can check the Katib Experiment info in the Katib UI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jovyan/.local/lib/python3.6/site-packages/kfp/dsl/_container_op.py:1028: FutureWarning: Please create reusable components instead of constructing ContainerOp instances directly. Reusable components are shareable, portable and have compatibility and support guarantees. Please see the documentation: https://www.kubeflow.org/docs/pipelines/sdk/component-development/#writing-your-component-definition-file The components can be created manually (or, in case of python, using kfp.components.create_component_from_func or func_to_container_op) and then loaded using kfp.components.load_component_from_file, load_component_from_uri or load_component_from_text: https://kubeflow-pipelines.readthedocs.io/en/latest/source/kfp.components.html#kfp.components.load_component_from_file\n",
      "  category=FutureWarning,\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<a href=\"/pipeline/#/experiments/details/608c6385-33ee-4a2f-81a1-35662f61028c\" target=\"_blank\" >Experiment details</a>."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<a href=\"/pipeline/#/runs/details/68100cc5-d42d-441e-974e-f61dd40782ef\" target=\"_blank\" >Run details</a>."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "RunPipelineResult(run_id=68100cc5-d42d-441e-974e-f61dd40782ef)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kfp.Client().create_run_from_pipeline_func(median_stop, arguments={})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
